Shape from periodic texture using the spectrogram

Texture has long been recognized in computer vision as an important monocular shape cue, with texture gradients yielding information on surface orientation. A more recent trend is the analysis of images in terms of local spatial frequencies, where each pixel has associated with it its own spatial frequency distribution. This has proven to be a successful method of reasoning about and exploiting many imaging phenomena. Thinking about both shape-from-texture and local spatial frequency, it seems that texture gradients would cause systematic changes in local frequency, and that these changes could be analyzed to extract shape information. However, there does not yet exist a theory that connects texture, shape, and the detailed behavior of local spatial frequency. We show in this paper how local spatial frequency is related to the surface normal of a textured surface. We find that the Fourier power spectra of any two similarly textured patches on a plane are approximately related to each other by an affine transformation. The transformation parameters are a function of the plane’s surface normal. We use this relationship as the basis of a new algorithm for finding surface normals of textured shapes using the spectrogram, which is one type of local spatial frequency representation. We validate the relationship by testing the algorithm on real textures. By analyzing shape and texture in terms of the local spatial frequency representation, we can exploit the advantages of the representation for the shape-from-texture problem. Specifically, our algorithm requires no feature detection and can give correct results even when the texture is aliased.